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We present a quantum information theory that allows for the consistent 
description of quantum entanglement. It parallels classical (Shannon) 
information theory but is based entirely on density matrices, rather 
than probability distributions, for the description of quantum ensem- 
bles. We find that, unlike in Shannon theory, conditional entropies can 
be negative when considering quantum entangled systems such as an 
Einstein-Podolsky-Rosen pair, which leads to a violation of well-known 
bounds of classical information theory. Negative quantum entropy can 
be traced back to "conditional" density matrices which admit eigenval- 
ues larger than unity. A straightforward definition of mutual quantum 
entropy, or "mutual entanglement", can also be constructed using a 
"mutual" density matrix. Such a unified information-theoretic descrip- 
tion of classical correlation and quantum entanglement clarifies the link 
between them: the latter can be viewed as "super- correlation" which 
can induce classical correlation when considering a ternary or larger 
system. 



1. INTRODUCTION 

Quantum information theory |1| is a new field with potential im- 
plications for the conceptual foundations of quantum mechanics. It ap- 
pears to be the basis for a proper understanding of the emerging fields 
of quantum computation ||, quantum communication |§, and quan- 
tum cryptography Q. Although some fundamental results have been 
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obtained recently such as the quantum noiseless coding theorem || or 
the rules governing the extraction of classical information from quan- 
tum entropy, it is still puzzling in many respects. Quantum information 
processing basically deals with quantum bits (qubits) rather than 
bits, the former obeying quantum laws quite different from the classical 
physics of bits that we are used to. Most importantly, qubits can exist 
in quantum superpositions, a notion completely inaccessible to classi- 
cal mechanics, or even classical thinking. To accommodate the relative 
phases in quantum superpositions, quantum information theory must 
be based on mathematical constructions which reflect these: the den- 
sity matrices. The central object of information theory, the entropy, 
has been introduced in quantum mechanics by von Neumann |J 

S (p) = -Trplogp . (1) 
Its relationship to the Boltzmann-Gibbs-Shannon entropy 

#(P) = ~J2Pi l °gPi ( 2 ) 

i 

is obvious when considering the von Neumann entropy of a mixture of 
orthogonal states, in which case the density matrix p in ([!]) contains 
classical probabilities Pi on its diagonal, and S(p) = H(p). In general, 
however, quantum mechanical density matrices have off-diagonal terms, 
which reflect the relative quantum phases in superpositions. 

In classical (Shannon) information theory the concept of con- 
ditional probabilities has given rise to the definition of conditional and 
mutual entropies. These can be used to elegantly describe the trade-off 
between entropy and information in measurement, as well as the char- 
acteristics of a transmission channel. For example, for two systems A 
and B, the measurement of A by B is expressed by the equation for 
the entropies 

H{A) — H(A\B) + H(A:B) . (3) 

Here, H(A\B) is the entropy of A after having measured those pieces 
that become correlated in B, while H(A:B) is the information gained 
about A via the measurement of B. Mathematically, H(A\B) is a con- 
ditional entropy, and is defined using the conditional probability Pi\j and 
the joint probability Pi j describing random variables from ensembles A 
and B: 

H(A\B) = -52 Pij log Pilj . (4) 
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The mutual entropy or information H(A:B), on the other hand, is 
defined via the mutual probability p i: j = PiPj/pij as 

H{A:B) = -Y^P^ogpr, . (5) 

Simple relations such as Py = Pi\jPj imply equations such as H(A\B) = 
H(AB) — H(B) and all the other usual relations of classical information 
theory [e.g., Eq. (|3|)]. Curiously, a quantum information theory par- 
alleling these constructions has never been attempted. Rather, a "hy- 
brid" theory was used in which quantum probabilities are inserted in 
the classical formulae given above, thereby loosing the quantum phase 
crucial to density matrices (see, e.g., ||). Below in Section 2 we show 
that a consistent quantum information theory can be developed that 
parallels the construction outlined above, while based entirely on ma- 
trices 



2. QUANTUM INFORMATION THEORY 

Let us consider the information-theoretic description of a compos- 
ite quantum system AB. A straightforward quantum generalization of 
Eq. (£|) suggests the definition 

S(A\B) = -Tr abIpab fog p A{B ] (6) 

for the quantum conditional entropy. In order for such an expression 
to hold, we define the concept of a "conditional" density matrix, 



Pa\b = Pa'bO-a ® Pb) 1,n X n -> oo , (7) 



the analog of the conditional probability pqj. Here, 1a is the unity 
matrix in the Hilbert space for A, ® stands for the tensor product in the 
joint Hilbert space, and pB = ^a[pab\ denotes a "marginal" density 
matrix, analogous to the marginal probability pj = J2iPij- The peculiar 
form involving the infinite limit in Eq. ([F]) is necessary because joint 
and marginal density matrices do not commute in general. However, 
the definition implies that the standard relation 

S{A\B) = S(AB) - S(B) (8) 

holds for the quantum entropies and that S(A\B) is invariant under any 
unitary transformation of the product form U A ® Ub- More precisely, 
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it is easy to see that pa\b is a positive Hermitian operator (in the joint 
Hilbert space) whose spectrum is invariant under Ua®Ub- Despite the 
apparent similarity between the quantum definition for S(A\B) and the 
standard classical one for H(A\B), dealing with matrices rather than 
scalars opens up a quantum realm for information theory exceeding the 
classical one. The crucial point is that, while pju is a probability distri- 
bution in i (in particular < pqj < 1), its quantum analog pa\b is not 
a density operator: it can have eigenvalues larger than one, and, con- 
sequently, the associated conditional entropy S(A\B) can be negative. 
Only such a matrix-based quantum formalism consistently accounts 
for the well-known non-monotonicity of quantum entropies (see, e.g., 
[ID"]). This means that it is acceptable, in quantum information theory, 



to have S(AB) < S(B), i.e., the entropy of the entire system AB can 
be smaller than the entropy of one of its subparts B, a situation which 
is of course forbidden in classical information theory. This happens for 
example in the case of quantum entanglement between A and B, and 
will be illustrated below for an EPR pair. Note that, as a consequence 
of the concavity of S(A\B), a property related to strong subadditivity 
(see, e.g., [[TTJ) any separable state (i.e., a mixture of product states) 
is associated with non-negative S(A\B). (The converse is not true.) 
Therefore, the non-negativity of conditional entropies can be viewed 
as a necessary condition for separability, and we have shown that this 
condition can be related to entropic Bell inequalities flTl |. 



Similarly, the quantum analog of the mutual entropy can be con- 
structed, defining a "mutual" density matrix 



Pa-.b 



(pA®PB) 1/n PAB n Y n^oo, (9) 



the analog of the mutual probability p i: j. As previously, this definition 
implies the standard relation 

S{A:B) = 3(A) + S(B) - S(AB) (10) 

between the quantum entropies. This definition extends the classical 
notion of mutual or correlation entropy H(A:B) to the quantum no- 
tion of mutual entanglement S(A:B) and applies to pure as well as 
mixed states; S(A:B) is a general measure of correlations and "super- 
correlations" in information theory. In fact, all the above quantum 
definitions reduce to the classical ones for a diagonal pab, which sug- 
gests that Eqs. fl7|) and ([|) are very reasonable assumptions. It is 
possible that other definitions of pa\b and Pa-.b could be proposed, but 
we believe this choice is simplest. This formalism suggests that all the 
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Figure 1: (a) General entropy diagram for a quantum composite system 
AB. (b) Entropy diagrams for three cases of a system of 2 qubits: (I) 
independent, (II) classically correlated, (III) quantum entangled. 



relations between classical entropies {e.g., the chain rules for entropies 
and mutual entropies) also have a quantum analog, and we make use 



of it in |5 0. 

The relations between entropies are conveniently summarized by 
a Venn-like entropy diagram, as shown in Fig. [TJa. The important dif- 
ference between classical and quantum entropy diagrams is that the 
basic inequalities relating the entropies are "weaker" in the quantum 
case, allowing for negative conditional entropies and "excessive" mu- 
tual entropies |§. For example, the upper bound for the mutual en- 
tropy (which is directly related to the channel capacity) is H(A:B) < 
mm[H (A) , H (B)] in classical information theory, while it can reach 
twice the classical upper bound S(A:B) < 2mm[S(A), S(B)] in quan- 
tum information theory as a consequence of the Araki-Lieb inequality 
(see, e.g., jnj). In Fig. |TJt>, we show the entropy diagram corresponding 
to three limiting cases of a composite system of two dichotomic vari- 
ables (e.g., 2 qubits): independent variables (case I), classically corre- 
lated variables (case II), and quantum entangled variables (case III). In 
all three cases, each subsystem taken separately is in a mixed state of 
entropy S(A) = S(B) = 1 bit. Cases I and II correspond to classical sit- 
uations (which can of course be described in our formalism with density 
matrices as well), while case III is a purely quantum situation which vio- 
lates the bounds of classical information theory [[| . It corresponds to an 
EPR pair, characterized by the pure state \iPab) = 2~ 1 / 2 (|01) — 1 10) ) , 
and, accordingly, it is associated with a vanishing combined entropy 
S(AB) = 0. Using qab = \'^ab){'4 } ab\, we see that subpart A (or B) 
has the marginal density matrix pa = Tt b[pab] = |(|0)(0| + |1)(1|), 
and is therefore in a mixed state of positive entropy. This purely quan- 
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turn situation corresponds to the unusual entropy diagram (-1,2,-1) 
shown in Fig. |l]b. That the EPR situation cannot be described classi- 
cally is immediately apparent when considering the conditional density 
matrix^: indeed the latter can be written as 



Pa\b = Pab(1a ® Pb) 






V o 





1 
-1 





0^ 
-1 

1 
0/ 



Plugging (JTI|) into definition (H) immediately yields S(A\B) = — 1. 
This is a direct consequence of the fact that pa\b has one "unclassical" 
(> 1) eigenvalue, 2. It is thus misleading to describe an EPR pair 
(or any of the Bell states) as a correlated state within Shannon the- 
ory, since negative conditional entropies are crucial to its description. 
In ||, we suggest that EPR pairs are better understood in terms of a 
qubit-antiqubit pair, where the qubit (antiqubit) carries plus (minus) 
one bit of information, and antiqubits are interpreted as qubits trav- 
eling backwards in time. Still, classical correlations (case II) emerge 
when observing an entangled EPR pair. Indeed, after measuring A, the 
outcome of the measurement of B is known with 100% certainty. The 
key to this discrepancy lies in the information-theoretic description of 



the measurement process JT2[ and will be briefly addressed in the next 
section. 



3. CORRELATION VERSUS ENTANGLEMENT 

The concept of negative conditional entropy turns out to be very 
useful to describe n-body composite quantum systems, and it sheds 
new light on the creation of classical correlations from quantum en- 
tanglement. Consider for example a 3-body system ABC in a GHZ 
state (or an "EPR-triplet"), \iPabc) = 2~ 1 / 2 (|000) + |111)). As it is 
a pure (entangled) state, the combined entropy is S(ABC) = 0. The 
corresponding ternary entropy diagram of ABC is shown in Fig. ||]a. 
Note that the ternary mutual entropy S(A:B:C) = S(A) + S(B) + 
S{C) - S(AB) - S{AC) - S{BC) + S(ABC) vanishes (see the cen- 
ter of the diagram); this is generic to any fully entangled three-body 
system. When tracing over the degree of freedom associated with 

■""Note that for Bell states, joint and marginal density matrices commute, simpli- 
fying definitions @) and (||). 



6 




(b) 




-1 



Figure 2: (a) Ternary entropy diagram for an "EPR-triplet" . (b) En- 
tropy diagram for subsystem AB unconditional on C. 

C, say, the resulting marginal density matrix for subsystem AB is 
Pab = TtcIpabc] — |(|00)(00| + |11)(11|), corresponding to a classically 
correlated system (case II). As the density matrix fully characterizes a 
quantum system, subsystem AB (unconditional on C, i.e., ignoring the 
existence of C) is in this case physically indistinguishable from a statis- 
tical ensemble prepared with an equal number of 1 00) and |11) states. 
Thus, A and B are correlated in the sense of Shannon theory if C is 
ignored. The "tracing over" operation depicted in Fig. 0d illustrates 
this creation of classical correlation from quantum entanglement. This 
feature is central to description the measurement process that we pro- 
pose in [12), where A and B represent two parts of the measurement 



device, while C is the measured quantum system. The subsystem AB 
unconditional on C has a positive entropy S(AB) = 1 bit, and is indis- 
tinguishable from a classical correlated mixture (this corresponds to the 
generation of random numbers). On the other hand, the entropy of C 
conditional on AB, S(C\AB), is negative and equal to —1 bit, thereby 
counterbalancing S(AB) to yield a vanishing combined entropy 

S(ABC) = S{AB) + S(C\AB) = . (12) 

as expected in view of the quantum entanglement between AB and C. 



We suggest in jl2| that this information-theoretic interpretation of en- 
tanglement paves the way to a natural, unitary, and causal model of 
the measurement process, devoid of any assumption of a wave-function 
collapse, while implying all the well-known results of conventional prob- 
abilistic quantum mechanics. The same framework can also be used to 
interpret the observation of classical correlation between the measure- 
ment devices that occurs in the measurement of an EPR pair, and sheds 
new light on quantum paradoxes 
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